Processing of Multimodal Retinal Images

ABSTRACT

There is provided a method of processing image data defining a fundus image of a retina to include supplementary information on a designated feature in the fundus image, comprising: designating (S10) a feature in the fundus image; receiving (S20) OCT data of a C-scan of the retina; selecting (S30) a subset of the OCT data representing a volumetric image of a part of the retina at a location on the retina corresponding to a location of the designated feature in the fundus image; processing (S40) the selected subset to generate, as the supplementary information, supplementary image data indicative of a variation, along a depth direction of the retina, of a measured reflectance of the eye in the selected subset; and combining (S50) the image data with the supplementary image data, such that an image defined by the combined data provides an indication of the variation at the designated feature.

TECHNICAL FIELD

Example aspects herein generally relate to the field of retinal imageprocessing and, more particularly, to the processing of a reflectanceimage of a retina to include supplementary information derived fromoptical coherence tomography (OCT) data acquired from the retina.

BACKGROUND

Two-dimensional images of the ocular fundus acquired by a fundus cameraor a scanning laser ophthalmoscope (SLO), for example, are widely usedto detect various eye diseases, as well as systemic diseases in asubject. Although these imaging methods tend to have a varyingsensitivity with depth in the retina, their depth specificity tends tobe low, so that the depth of an observed feature in a retinal fundusimage is usually uncertain. As some common eye diseases, such asage-related macular degeneration, diabetic retinopathy and retinal veinocclusion, are associated with features such as hyper-reflective dotswhich furthermore have a similar appearance in retinal fundus image, itis often difficult for a clinician to distinguish between these diseasesfrom an inspection of a retinal fundus image alone. Artefacts in retinalfundus images, such as those resulting from a highly reflective innerlimiting membrane (ILM), and atrophy of the retinal pigment epithelium(RPE) may also confuse the identification of disease-related features,such as exudative, RPE-related and drusenoid hyper-reflective dots.Further information on a feature of interest that has been identified ona retinal fundus image may be obtained by examining an OCT thickness mapwhich covers the feature of interest.

SUMMARY

There is provided, in accordance with a first example aspect herein, acomputer-implemented method of processing image data defining a fundusimage of a portion of a retina of an eye to include supplementaryinformation on a designated feature in the fundus image. The methodcomprises designating a feature in the fundus image, receiving opticalcoherence tomography (OCT) data of a C-scan of the portion of theretina, and selecting a subset of the OCT data which represents avolumetric image of a part of the retina at a location on the retinacorresponding to a location of the designated feature in the fundusimage. The method further comprises processing the selected subset ofthe OCT data to generate, as the supplementary information,supplementary image data indicative of a variation, along a depthdirection of the retina, of a measured reflectance of the eye in theselected subset of the OCT data. The method further comprises generatingcombined image data by combining the image data with the supplementaryimage data, such that a combined image defined by the combined imagedata provides an indication of the variation of the measured reflectanceof the eye in the selected subset of the OCT data at the designatedfeature.

In the computer-implemented method according to the first exampleaspect, the combined image data may be generated by replacing pixelvalues of a subset of pixels of the fundus image data with pixel valuesof the supplementary image data, such that a combined image defined bythe combined image data provides an indication of the variation of themeasured reflectance of the eye in the selected subset of the OCT dataat the designated feature. In some example embodiments, the combinedimage data may be generated by replacing pixel values of a subset ofpixels of the fundus image data, which subset of pixels defines asubregion of the fundus image which is at the location of the designatedfeature in the fundus image, with pixel values of the supplementaryimage data, such that a combined image defined by the combined imagedata provides an indication of the variation of the measured reflectanceof the eye in the selected subset of the OCT data at the designatedfeature. In some other example embodiments, the supplementary image datamay define a graphic which is indicative of the variation, along thedepth direction of the retina, of the measured reflectance of the eye inthe selected subset of the OCT data or, more specifically, how themeasured reflectance of the eye, as indicated in the selected subset ofthe OCT data varies along a depth direction of the retina. In theseother example embodiments, the combined image data may be generated byreplacing pixel values of a subset of pixels of the fundus image datawith pixel values of the supplementary image data such that a combinedimage defined by the combined image data comprises the graphic, which isoverlaid on the fundus image so as to provide an indication of thevariation of the measured reflectance of the eye in the selected subsetof the OCT data at the designated feature.

Additionally or alternatively, the computer-implemented method accordingto the first example aspect may, in accordance with an exampleembodiment, further comprise causing the fundus image and a cursor to bedisplayed on a display, such that the cursor can be controlled to moveover the displayed fundus image by a signal from a user input device,wherein the feature in the fundus image is designated by recording, inresponse to a feature designation command, a value of a first locationindicator that is indicative of a display location of the cursor on thedisplayed fundus image.

The method of the example embodiment may further comprise: processingthe OCT data to generate an OCT en-face image of the portion of theretina; and causing the OCT en-face image to be displayed on the displaytogether with the fundus image, such that the cursor can be controlledto move over the displayed OCT en-face image by the signal from the userinput device, wherein the subset of the OCT data is selected based on avalue of a second location indicator, which is indicative of a displaylocation of the cursor when the cursor has been guided by the signalfrom the user input device to overlay a part of the displayed OCTen-face image which corresponds to the designated feature in thedisplayed fundus image.

The feature in the fundus image may alternatively be designatedautomatically by a feature extraction algorithm. In this case, thecomputer-implemented method may further comprise: causing the fundusimage and a feature location indicator, which indicates a location ofthe designated feature in the fundus image, to be displayed on adisplay; processing the OCT data to generate an OCT en-face image of theportion of the retina; and causing the OCT en-face image and a cursor tobe displayed on the display together with the fundus image, such thatthe cursor can be controlled to move over the displayed OCT en-faceimage by a signal from a user input device, wherein the subset of theOCT data is selected based on a value of a second location indicator,which is indicative of a display location of the cursor when the cursorhas been guided by the signal from the user input device to overlay apart of the displayed OCT en-face image whose location corresponds tothe location of the designated feature in the fundus image that isindicated by the feature location indicator.

Alternatively, where the feature in the fundus image is designatedautomatically by a feature extraction algorithm, the subset of the OCTdata may be selected by applying a geometric transformation, which mapslocations in the fundus image to corresponding A-scan locations in theOCT data, to the location of the feature in the fundus image which hasbeen designated by the feature extraction algorithm.

In any of the computer-implemented methods set out above, the featuremay be one of a dot and a hyper-reflective dot in the fundus image, thefeature having a pathological cause or being caused by a reflection froman inner limiting membrane of the retina. For example, the feature mayhave a pathological cause comprising one of blood leakage, exudation,drusen, atrophy and/or naevi in the retina, and atrophy of a retinalpigment epithelium in the retina.

In the foregoing, the selected subset of the OCT data may be processedto generate the supplementary image data by the following scheme 1,scheme 2 or scheme 3.

Scheme 1:

The selected subset of the OCT data is processed to generate thesupplementary image data by: detecting a plurality of anatomical layersof the eye in the selected subset of the OCT data, the anatomical layerscomprising one or more retinal layers of the retina; calculating, foreach of at least two of the detected anatomical layers, a respective sumvalue by summing values of data elements of the subset of the OCT datain the anatomical layer; calculating, for each of the at least two ofthe detected anatomical layers, a respective ratio between the sum valuecalculated for the anatomical layer and a sum of all the data elementsthat are in the at least two of the detected anatomical layers and inthe subset of the OCT data; and generating, as the supplementary imagedata, and based on an ordered sequence of the calculated ratios, whereinthe calculated ratios are arranged in order of the correspondinganatomical layers in the eye, colour information defining a colour whichis to be displayed in the combined image and identifies the orderedsequence of the calculated ratios, such that the colour is indicative ofthe variation, along the depth direction of the retina, of the measuredreflectance of the eye in the selected subset of the OCT data. Forexample, three anatomical layers in the selected subset of the OCT datamay be detected, and the colour information may be generated byassigning, to each of a red colour component, a green colour componentand a blue colour component of the colour to be displayed in thecombined image, a respective weighting for the colour component inaccordance with a respective one of the calculated ratios in the orderedsequence of the calculated ratios.

Scheme 2:

The selected subset of the OCT data is processed to generate thesupplementary image data by: detecting a plurality of anatomical layersof the eye in the selected subset of the OCT data, the anatomical layerscomprising one or more retinal layers of the retina; calculating, foreach anatomical layer of the detected anatomical layers, a respectivesum value by summing values of data elements of the subset of the OCTdata in the anatomical layer; selecting, based on the calculated sumvalues, an anatomical layer of the detected anatomical layers whichprovides a dominant contribution to the measured reflectance of the eye;and generating, as the supplementary image data, graphic image datadefining a graphic which identifies the selected anatomical layer.

Scheme 3:

The selected subset of the OCT data represents a volumetric image of apart of the retina having a feature of a predetermined type and isprocessed to generate the supplementary image data by: training a modelfor determining a depth of the feature of the predetermined type in thedepth direction of the retina, by supervised learning of examples of OCTdata of pathological regions of at least one other retina, each of theexamples of OCT data comprising a single OCT A-scan or two or moreadjacent OCT A-scans, and each of the pathological regions having arespective feature of the predetermined type, wherein an indication of arespective depth of the respective feature in the depth direction of theretina in each of the examples of OCT data is specified by a user duringthe training; processing the selected subset of the OCT data using thetrained model to determine the depth of the feature in the depthdirection of the retina; and generating, as the supplementary imagedata, one of (i) graphic image data defining a graphic which indicatesthe determined depth of the feature and is to be overlaid on the fundusimage (10) so as to indicate a location of the feature in the combinedimage (40), and (ii) colour information defining a colour which is to bedisplayed at a location of the feature in the combined image andindicates the determined depth of the feature.

There is also provided, in accordance with a second example aspectherein, a computer program comprising computer program instructionswhich, when executed by a computer, cause the computer to perform themethod set out above. The computer program may be stored on anon-transitory computer-readable storage medium, or it may be carried bya signal.

There is also provided, in accordance with a third example aspectherein, an apparatus for processing image data defining a fundus imageof a portion of a retina of an eye to include supplementary informationon a designated feature in the fundus image. The apparatus comprises afeature designation module arranged to designate a feature in the fundusimage, and a receiver module arranged to receive OCT data of a C-scan ofthe portion of the retina. The apparatus further comprises a selectionmodule arranged to select a subset of the OCT data which represents avolumetric image of a part of the retina at a location on the retinacorresponding to a location of the designated feature in the fundusimage, and a supplementary image data generation module arranged toprocess the selected subset of the OCT data to generate, as thesupplementary information, supplementary image data indicative of avariation, along a depth direction of the retina, of a measuredreflectance of the eye in the selected subset of the OCT data. Theapparatus further comprises a combined image data generation modulearranged to generate combined image data by combining the image datawith the supplementary image data, such that a combined image defined bythe combined image data provides an indication of the variation at thedesignated feature.

In the apparatus according to the third example aspect, the combinedimage data generation module may be arranged to generate the combinedimage data by replacing pixel values of a subset of pixels of the fundusimage data with pixel values of the supplementary image data, such thata combined image defined by the combined image data provides anindication of the variation of the measured reflectance of the eye inthe selected subset of the OCT data at the designated feature. In someexample embodiments, the combined image data generation module may bearranged to generate the combined image data by replacing pixel valuesof a subset of pixels of the fundus image data, which subset of pixelsdefines a subregion of the fundus image which is at the location of thedesignated feature in the fundus image, with pixel values of thesupplementary image data, such that a combined image defined by thecombined image data provides an indication of the variation of themeasured reflectance of the eye in the selected subset of the OCT dataat the designated feature. In some other example embodiments, thesupplementary image data generation module may be arranged to generatesupplementary image data which defines a graphic indicative of thevariation, along the depth direction of the retina, of the measuredreflectance of the eye in the selected subset of the OCT data or, morespecifically, how the measured reflectance of the eye, as indicated inthe selected subset of the OCT data varies along a depth direction ofthe retina. In these other example embodiments, the combined image datageneration module may be arranged to generate the combined image data byreplacing pixel values of a subset of pixels of the fundus image datawith pixel values of the supplementary image data such that a combinedimage defined by the combined image data comprises the graphic, which isoverlaid on the fundus image so as to provide an indication of thevariation of the measured reflectance of the eye in the selected subsetof the OCT data at the designated feature.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will now be explained in detail, by way ofnon-limiting example only, with reference to the accompanying figuresdescribed below. Like reference numerals appearing in different ones ofthe figures can denote identical or functionally similar elements,unless indicated otherwise.

FIG. 1 is a schematic illustration of an apparatus for processing imagedata, according to a first example embodiment.

FIG. 2 is a schematic illustration of a display displaying a fundusimage and an en-face OCT image in first example embodiment.

FIG. 3 is a schematic illustration OCT data and a subset thereof whichis selected by the selection module of the first example embodiment.

FIG. 4 illustrates an example implementation in programmable signalprocessing hardware of the first of the example embodiment herein.

FIG. 5 is a flow diagram illustrating a process by which the apparatusof the first example embodiment processes image data defining a fundusimage of a portion of a retina of an eye to include supplementaryinformation on a designated feature in the fundus image.

FIG. 6 is a flow diagram illustrating a process by which thesupplementary image data generation module of the first exampleembodiment generates supplementary image data.

FIG. 7 shows a fundus image (image A), a processed version of the fundusimage in which designated features are highlighted (image B), and acombined image (image C) generated by the apparatus of the first exampleembodiment.

FIG. 8 shows an image of a B-scan taken along the horizontal line shownin image C of FIG. 7.

FIG. 9 is a schematic illustration of an apparatus for processing imagedata, according to a second example embodiment.

FIG. 10 is a flow diagram illustrating a process by which thesupplementary image data generation module of the second exampleembodiment generates supplementary image data.

FIG. 11 shows a first example of a combined image generated by theapparatus of the second example embodiment.

FIG. 12 shows a second example of a combined image generated by theapparatus of the second example embodiment.

FIG. 13 is a schematic illustration of an apparatus for processing imagedata, according to a third example embodiment.

FIG. 14 is a flow diagram illustrating a process by which thesupplementary image data generation module of the third exampleembodiment generates supplementary image data.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

There are described in the following a method, apparatus and computerprogram for processing retinal fundus image data that may assist aclinician to assess a distribution of, and distinguish between, featuresof some common eye diseases which have similar appearance in retinalfundus images, such as diabetic retinopathy, age-related maculardegeneration and retinal vein occlusion, and that may help avoidmisreading of retinal fundus images caused by specular imagingartefacts. The techniques described herein may allow additionalinformation in OCT data to be leveraged for a clearer rendering offeatures (e.g. bright spots) in the retinal fundus images. In someexample embodiments, the information can be assimilated by the userwithout reviewing OCT data. Furthermore, in some example embodiments,the user may be notified of whether a feature in a retinal fundus imageis coincident with a feature in OCT, thereby helping to avoidmisinterpretation of artefactual spots and the like in the retinalfundus image.

Example embodiments herein will now be described in more detail withreference to the accompanying drawings.

First Example Embodiment

FIG. 1 is a schematic illustration of an apparatus 100 for processingimage data according to a first example embodiment. As will be describedin more detail below, the apparatus 100 is arranged to process receivedimage data D_(F), which defines a fundus image 10 of a portion of aretina of an eye, to include supplementary information on a designatedfeature 12 in the fundus image 10, as illustrated in FIG. 2, where thefundus image 10 and the designated feature 12 are shown to be displayedon a display 14 (e.g. a visual display unit, such as a computermonitor), together with a user-controlled cursor 16.

The fundus image of the retina (also referred to herein as a retinalfundus image) 10 may be acquired by any fundus imaging process by whicha two-dimensional representation of the three-dimensional(semi-transparent) retinal tissues projected onto an imaging plane of afundus imaging apparatus (not shown) is obtained using light collectedfrom the retina. The fundus image may be acquired by illumination of theretina by global illumination or by a scanned light beam provided by thefundus imaging apparatus. The light reflected from the retina iscollected by a receiver of the fundus imaging apparatus, and itsposition-dependent intensity is detected, and then converted into theimage data D_(F) representing the two-dimensional fundus image 10. Thus,the term ‘fundus imaging’ used herein refers to any process whichresults in a two-dimensional image of a portion of the retina, whereinimage pixel values represent respective intensities of light collectedfrom the retina, and is to be contrasted with OCT imaging (discussedbelow).

The fundus image 10 may be acquired by one of many types of fundusimaging apparatus known to those versed in the art, including but notlimited to fundus cameras and scanning laser ophthalmoscopes (SLOs). Incombination with light filters, these types of fundus imaging apparatusmay be used to acquire monochromatic or autofluorescence images of thefundus and, with the injection of intravenous contrast material, mayalso be used to acquire fundus fluorescein angiograms, indocyanine greenangiograms and the like. Fundus imaging thus covers variousmodalities/techniques, including monochromatic fundus photography,colour fundus photography, scanning laser ophthalmoscopy (SLO), adaptiveoptics SLO, fluorescein angiography and indocyanine angiography.

The fundus imaging apparatus may have a relatively narrow field of viewor 30°-55° or so that is typical in conventional fundus imaging, or itmay be a widefield fundus imaging apparatus having a field of view ofabout 100°. As a further alternative, the fundus imaging apparatus maybe an ultra-widefield (UWF™) fundus imaging apparatus having a field ofview of about 200°, such as the Optos California™ system made by Optosplc.

As shown in FIG. 1, the apparatus 100 comprises a feature designationmodule 110, which is arranged to designate a feature 12 in the fundusimage 10 received from the fundus imaging apparatus. The feature 12 maybe an image of a structure having a relatively high reflectivity(whether inside the retina or in another part of the eye being imaged),and may therefore be a part of the fundus image 10 representing higherreflectance than a region of the fundus image 10 surrounding that part.The feature 12 may alternatively be an image of a structure having arelatively low reflectivity (whether inside the retina or in anotherpart of the eye being imaged), and may therefore be a part of the fundusimage 10 representing lower reflectance than a region of the fundusimage 10 surrounding that part. Although the description of theapparatus 100 is described herein by reference to a single designatedfeature 12 for convenience, it will be appreciated that multiplefeatures in the fundus image 10 may be designated, and the operationsdescribed in the following may be performed on the basis of each ofthese designated features.

The feature 12 may, for example, be a dot feature, whose extent in thefundus image is small compared to the size of the fundus image. Dotfeatures of this kind can have a pathological cause, such as bloodleakage, exudation, drusen, atrophy and/or naevi in the retina. Dotfeatures related to these pathologies have a very similar,characteristic appearance (in terms of their sizes and the brightnessdifference between dot features and their surroundings in the fundusimage 10), and are difficult for a clinician to disambiguate byinspection of the fundus image 10 alone. They are also difficult todisambiguate from dot features having non-pathological causes, such asfloaters in the ocular vitreous, reflex from retinal surfaces such asthe inner limiting membrane, or reflections from imperfections such asdust in the imaging system, for example, which have a similar appearancein fundus images to the aforementioned pathological dot features.

The dot feature may be a hyper-reflective dot/focus, for example, whichis often observed in retinal fundus images. Hyper-reflective foci mayhave a pathological cause such as blood leakage, exudation, drusen,atrophy and/or naevi in the retina or atrophy of a retinal pigmentepithelium in the retina, or a non-pathological cause such as areflection from an inner limiting membrane of the retina.Hyper-reflective dots in all these cases have a very similar,characteristic appearance, and are difficult to distinguish byinspection of the fundus image 10 alone.

The feature designation module 110 may designate the feature 12 in thefundus image 10 in one of a number of different ways. For example, inthe present example embodiment, the feature designation module 110causes the fundus image 10 and the cursor 16 to be displayed on thedisplay 14. The display location of the cursor 16 on the displayedfundus image 10 is controllable by a signal from a user input device 18,such as a computer mouse, a trackpad or the like. In the present exampleembodiment, the feature 12 in the fundus image 10 is designated byrecording, in response to a feature designation command, a value of afirst location indicator that is indicative of a display location of thecursor 16 when the cursor has been guided by the signal to overlay thedisplayed fundus image 10. The feature designation module 110 may causea graphic 19 to be overlaid on the reluctance image 10 at the locationin the fundus image 10 indicated by the first location indicator.

The feature designation command may, as in the present exampleembodiment, be provided by the user, for example by the user operatingthe user input device 18 (e.g. providing a mouse click in case the userinput device is provided in the form of a computer mouse, or a fingertap in case the user input device is provided in the form of atrackpad). The feature designation command may alternatively be providedby the user operating another user input device, for example pressing akey on a computer keyboard. As a further alternative, the featuredesignation command may be generated by the feature designation module110 in response to a prescribed condition being met, for example theexpiry of a predetermined time period which is started by a display onthe display 14 of an instruction to the user to designate a feature ofinterest in the displayed fundus image 10.

The feature designation module 110 need not rely on such a humaninteraction to designate the feature 12, and may, in an alternativeexample embodiment, perform the designation automatically, using one ofa number of feature extraction algorithms known to those versed in theart, for example as described in “Automated detection of age-relatedmacular degeneration in color fundus photography: a systematic review”by Pead, E., Megaw, R., Cameron, J., Fleming, A., Dhillon, B., Trucco,E. and MacGillivray, T., 2019, published in Survey of Ophthalmology,64(4), 2019 at pages 498-511, or “A review on exudates detection methodsfor diabetic retinopathy” by Joshi, S. and Karule, P. T., published inBiomedicine & Pharmacotherapy, 97, 2018, at pages 1454-1460, or “Areview on recent developments for detection of diabetic retinopathy” byAmin, J., Sharif, M. and Yasmin, M., Scientifica, 2016.

The apparatus 100 also has a receiver module 120, which is arranged toreceive optical OCT data D_(OCT) of a C-scan (shown at 20 in FIG. 3) ofthe portion of the retina acquire by an OCT imaging system (not shown).The apparatus 100 also has a selection module 130, which is arranged toselect a subset d_(OCT) of the OCT data D_(OCT), which represents avolumetric image (defined by voxels of a set 22 of A-scans in FIG. 3) ofa part of the retina at a location on the retina corresponding to alocation of the designated feature 12 in the fundus image 10. Asillustrated schematically in FIG. 3, the C-scan 20 is composed of aseries of B-scans 24, wherein each B-scan 24 is composed of a series ofA-scans 26. The value of each data element in an A-scan 26 provides anindication of a measured reflectance of the eye at a respective locationin the depth direction of the retina that corresponds to the location ofthe data element along the A-scan 26 (i.e. along the z-axis direction inFIG. 3). The location of the A-scan 26 along the y-axis and x-axisdirections of data element array forming the C-scan 20 corresponds to alocation on the retina of a measurement taken by the OCT imaging systemto acquire the A-scan 26.

The OCT imaging system used to acquire the OCT data D_(OCT) may be ofany type known to those versed in the art, for example a point-scan OCTimaging system, which can acquire an OCT image by scanning a laser beamlaterally across a region of the eye. The OCT imaging system mayalternatively be a parallel acquisition OCT imaging system, such asFull-Field OCT (FF-OCT) or Line-Field OCT (LF-OCT), which may offersuperior A-scan acquisition rates (up to tens of MHz) by illuminating anarea or a line on the sample, rather than scanning a single spot acrossthe eye. In FF-OCT, a two-dimensional region of the eye is illuminatedat the same time and the lateral positions across the region areconcurrently captured using a photodetector array such as a high-speedcharge-coupled device (CCD) camera. Where the OCT imaging system is aFull-field OCT, it may take the form of a full-field time-domain OCT(FF-TD-OCT) or full-field swept-source OCT (FF-SS-OCT), for example. InFF-TD-OCT, the optical length of the reference arm can be varied duringa scan in order to image regions at different depths in the eye. Eachframe captured by the high-speed camera in FF-TD-OCT thereforecorresponds to a slice of the eye at a respective depth within the eye.In FF-SS-OCT, the sample region is full-field illuminated using a sweptlight source that emits light whose wavelength varies over time. As thewavelength of the swept light source is swept over a range of opticalwavelengths, a spectrogram correlating reflectivity information againstoptical wavelength can be generated by the high-speed camera for eachcamera pixel. Each frame captured by the camera therefore corresponds toreflectivity information for a single wavelength of the swept lightsource. Upon acquiring a frame for every wavelength of the swept lightsource, a C-scan of the region can be obtained by performing a Fouriertransform on the samples of spectrograms generated by the camera. Inline-field OCT (LF-OCT), a line of illumination may be provided to thesample and a B-scan may be acquired in the imaging process. Line-fieldOCT may be classified as line-field time-domain OCT (LF-TD-OCT),line-field swept-source OCT (LF-SS-OCT), or line-field spectral-domainOCT (LF-SD-OCT), for example.

The OCT imaging system used to acquire the OCT data and the fundusimaging apparatus used to acquire the fundus image may be separatedevices. It should be noted, however, that the fundus image and the OCTdata may be acquired by a single, multimodal retinal imaging system,such as the Silverstone combined UWF retinal imaging device andUWF-guided swept source OCT scanner made by Optos plc.

The subset d_(OCT) of the received OCT data D_(OCT) may, as in thepresent example embodiment, be selected by the selection module 130based on an input from a user who has inspected a two-dimensionalrepresentation of the OCT data D_(OCT) shown on the display 14,determined a location in the representation of the OCT data D_(OCT)which corresponds to the location of the designated feature 12 in thereference image 10 also shown on the display 14, and guided the cursor16 to overlay the determined location in the representation of the OCTdata D_(OCT).

More particularly, in the present example embodiment, the selectionmodule 130 processes the received OCT data D_(OCT) to generate an OCTen-face image of the portion of the retina. The OCT en-face image 30 isa projection of the (three-dimensional) C-scan 20 image to the sametwo-dimensional plane that would be viewed in the fundus image 10 of thesame portion of the retina. The generation of the OCT en-face image 30may involve a summation, a weighted summation or a maximisation of thedata elements (voxels) of the C-scan along the depth axis (z) of the OCTC-scan 20.

The selection module 130 causes the OCT en-face image 30 to be displayedon the display 14 together with the fundus image 10, for examplealongside the fundus image 10 as illustrated in FIG. 2, with the cursor16 being controllable by the signal from the user input device 18 tomove over the displayed OCT en-face image 30. Although the display 14 isprovided in the form of a single visual display unit in the presentexample embodiment, the display 14 may alternatively be provided in theform of a first monitor (or other visual display unit) and a secondmonitor (or other visual display unit, not necessarily of the same kindas the first visual display unit), which are arranged to respectivelydisplay the fundus image 10 and the OCT en-face image 30 simultaneouslyto the user.

In the present example embodiment, the subset d_(OCT) of the OCT dataD_(OCT) is selected by the selection module 130 based on a value of asecond location indicator that is indicative of a display location ofthe cursor 16 when the cursor 16 has been guided by the signal from theuser input device 18 to overlay a part 32 of the displayed OCT en-faceimage 30 which is judged by the user (based on a comparison of retinalfeatures in the displayed images 10 and 30) to correspond to thedesignated feature 12 in the displayed fundus image 10, and in responseto a feature designation command of the kind described above. The subsetd_(OCT) of the OCT data D_(OCT) may be selected based on the value of asecond location indicator by mapping the value of the second locationindicator to an A-scan of the OCT C-scan 20 having a corresponding (x,y) coordinate, and selecting the A-scan, or the A-scan together with apredefined arrangement of neighbouring (adjacent) A-scans (e.g. madjacent A-scans that are nearest to the mapped to- and selected A-scanin the x-y plane of the C-scan 20, where m is an integer, preferablygreater than 4 and more preferably greater than 8), as the subsetd_(OCT) of the OCT data D_(OCT).

It should be noted, however, that the subset d_(OCT) of the OCT dataD_(OCT) need not be defined by the mapped to- and selected A-scantogether with a predefined arrangement of neighbouring A-scans but mayalternatively be defined by a set of A-scans that are enclosed by acontour in the x-y plane of the C-scan 20 defined by straight-linesegments linking A-scans of the C-scan 20 that have been selected basedon respective values of the second location indicator, in response tothe user issuing multiple feature designation commands as he/she movesthe cursor 16 around the boundary of a feature of interest in the fundusimage 10. The subset d_(OCT) of the OCT data D_(OCT) may thereforecorrespond to a region of the OCT en-face image 30 having a predefinedshape (as in the present example embodiment) or a shaped defined by theuser.

The selection module 130 may cause a graphic 34 to be overlaid on theOCT en-face image 30 at the location in the en-face image 30 indicatedby the second location indicator. The graphic 34 may the same as thegraphic 19, or different from graphic 19 in case a different colourand/or shape for the graphic 34 provides better visibility on the OCTen-face image 30, for example. Similarly, the appearance of the cursor16 whilst overlaying the displayed fundus image 10 may be different (interms of shape and/or colour) from its appearance whilst overlaying thedisplayed OCT en-face image 30.

In the alternative example embodiment identified above, in which thefeature designation module 110 designates the feature 12 in the fundusimage 10 automatically by use of a feature extraction algorithm, theselection module 130 may cause a feature location indicator (e.g. in theform of graphic 19) to be overlaid on the displayed fundus image 10, soas to indicate the location of the designated feature 12 in the fundusimage 10. In this case, the subset d_(OCT) of the OCT data D_(OCT) maybe selected based on a value of the second location indicator asdescribed above, which is indicative of a display location of the cursor16 when the cursor 16 has been guided by the signal from the user inputdevice 18 to overlay a part of the displayed OCT en-face image 30 whoselocation is judged by the user (based on a comparison of retinalfeatures in the displayed images 10 and 30) to correspond to thelocation in the fundus image 10 indicated by the feature locationindicator.

It should be noted, however, that the correspondence between thelocation of the designated feature 12 in the fundus image 10 and thelocation of the subset door within OCT data D_(OCT) need not bedetermined by a user from an inspection of a displayed fundus image 10and a displayed OCT en-face image of a common portion of the retina. Insome example embodiments, the location of the subset d_(OCT) within OCTdata D_(OCT) may be determined automatically, on the basis of thelocation of the designated feature 12 in the fundus image 10, by use ofa predetermined geometric transformation. The predetermined geometrictransformation maps locations in the fundus image 10 to correspondingA-scan locations in the OCT data, and may be applied by the selectionmodule 130 to the location of the feature 12 in the fundus image 10 thathas been designated by the feature extraction algorithm, in order toidentify a corresponding A-scan location in the C-scan 20 of an A-scan26 which is to be included in the subset d_(OCT) of the OCT data(optionally, together with one or more adjacent A-scans in the C-scan20).

The geometric transformation may be determined in one of a number ofdifferent ways. For example, the geometric transformation may bedetermined by registering image data D_(F) defining the fundus image 10with image data defining an OCT en-face image 30 that has been generatedas described above, without the OCT en-face image 30 being displayed tothe user. Any intensity- and/or feature-based registration algorithmknow to those versed in the art may be used to determine the geometrictransformation and thus establish a point-to-point correspondencebetween locations in the fundus image 10 and lateral positions (definedby coordinates along the x- and y-axes in FIG. 3) in the OCT C-scan 20.It should be noted that such registration algorithms may operatedirectly on the two-dimensional dataset defining the fundus image 10 andthe three-dimensional dataset defining the OCT C-scan 20, rather than onthe two-dimensional dataset defining the fundus image 10 and thetwo-dimensional dataset of the OCT en-face image 30 that is derived fromthe three-dimensional dataset of the OCT C-scan 20.

In some example embodiments, both the designation of a feature in thefundus image 10 by the feature designation module 110, and the selectionof the subset d_(OCT) of the OCT data D_(OCT) by the selection module130, may be performed automatically (i.e. without any user input). Thus,in such example embodiments, the feature designation module 110 maydesignate a part of the image data D_(F) defining a feature 12 in thefundus image 10 using a feature extraction algorithm, and the selectionmodule 130 may select the subset d_(OCT) of the OCT data D_(OCT) byapplying the geometric transformation described above to the location ofthe designated part in the image data D_(F) of the fundus image 10. Inthese example embodiments, the designation of the feature 12 and theselection of the subset d_(OCT) of the OCT data is performed withoutdisplaying any representation of the image data D_(F) of the fundusimage 10 or of the OCT data D_(OCT) to the user, and without requiringany user input.

Referring again to FIG. 1, the apparatus 100 also has a supplementaryimage data generation module 140-1, which is arranged to process theselected subset d_(OCT) of the OCT data D_(OCT) to generate, as thesupplementary information, supplementary image data D_(SI) indicative ofa variation, along the depth direction of the retina (i.e. the directionin which light rays enter the retina), of a measured reflectance of theeye in the selected subset d_(OCT) of the OCT data. The apparatus 100also has a combined image data generation module 150, which is arrangedto generate combined image data D_(CI) by combining the image data D_(F)with the supplementary image data D_(SI), such that a combined imagedefined by the combined image data D_(CI) provides an indication of thevariation of the measured reflectance of the eye in the selected subsetd_(OCT) of the OCT data D_(OCT) at the designated feature. Thefunctionality of the supplementary image data generation module 140-1and the combined image data generation module 150 is described in moredetail below.

FIG. 4 is a schematic illustration of a programmable signal processinghardware 200, which may be configured to perform the operations of theapparatus 100 of the first example embodiment.

The programmable signal processing apparatus 200 includes acommunication interface (I/F) 210, for communicating with the fundusimaging apparatus and the OCT imaging system (or with the combinedimaging system mentioned above, which is capable of generating both theimage data D_(F) defining the fundus image 10 and the OCT data D_(OCT)of a subject retina) to receive the image data D_(F) of the fundus image10 and the OCT data D_(OCT) therefrom. The signal processing apparatus200 further includes a processor (e.g. a Central Processing Unit, CPU)220, a working memory 230 (e.g. a random access memory) and aninstruction store 240 storing a computer program 245 comprisingcomputer-readable instructions which, when executed by the processor220, cause the processor 220 to perform various functions of theapparatus 100 described herein. The working memory 230 storesinformation used by the processor 220 during execution of the computerprogram 245. The instruction store 240 may include a ROM (e.g. in theform of an electrically-erasable programmable read-only memory (EEPROM)or flash memory) which is pre-loaded with the computer-readableinstructions. Alternatively, the instruction store 240 may include a RAMor similar type of memory, and the computer-readable instructions of thecomputer program 245 can be input thereto from a computer programproduct, such as a non-transitory, computer-readable storage medium 250in the form of a CD-ROM, DVD-ROM, etc. or a computer-readable signal 260carrying the computer-readable instructions. In any case, the computerprogram 245, when executed by the processor 220, causes the processor220 to execute a method of processing the image data to include thesupplementary information described herein. It should be noted, however,that the apparatus 100 may alternatively be implemented innon-programmable hardware, such as an application-specific integratedcircuit (ASIC).

FIG. 5 is a flow diagram illustrating a method by which the apparatus100 of FIG. 1 processes the image data D_(F) defining the fundus image10 to include supplementary information on a designated feature 12 inthe fundus image 10.

In process S10 of FIG. 5, the feature designation module 110 designatesa feature 12 in the fundus image 10 of a portion of a retina that isdefined by image data D_(F) received from a fundus imaging apparatus, asdescribed above.

In process S20 of FIG. 5, the receiver module 120 receives OCT dataD_(OCT) of an OCT C-scan of the portion of the retina, as describedabove. Although process S20 is illustrated to occur following processS10 in FIG. 5, it will be appreciated that the order in which theseprocesses are performed may be reversed, and that at least parts ofthese processes may be performed concurrently.

In process S30 of FIG. 5, the section module 130 selects a subsetd_(OCT) of the received OCT data D_(OCT) which represents a volumetricimage of a part of the retina at a location on the retina correspondingto a location of the designated feature 12 in the fundus image 10, asdescribed above.

In process S40 of FIG. 5, the supplementary image data generation module140-1 processes the selected subset door of the OCT data D_(OCT) togenerate, as the supplementary information, supplementary image dataD_(SI) that is indicative of a variation, along a depth direction of theretina, of a measured reflectance of the eye in the selected subsetd_(OCT) of the OCT data D_(OCT). In other words, the supplementary imagedata Ds, indicates how the measured reflectance of the eye, as indicatedby the voxels in the selected subset d_(OCT) of the OCT data D_(OCT),varies along the depth direction of the retina. The supplementary imagedata D_(SI) may chart the variation of the measured reflectance of theeye, as indicated by the voxels in the selected subset d_(OCT) of theOCT data D_(OCT), with position along the depth direction of the retina.The supplementary image data D_(SI) may thus provide a profile of themeasured reflectance along the depth direction of the retina.

In the present example embodiment, the supplementary image datageneration module 140-1 processes the selected subset d_(OCT) of the OCTdata D_(OCT) to generate the supplementary image data D_(SI) by a methodwhich will now be described with reference to FIG. 6.

In process S42 of FIG. 6, the supplementary image data generation module140-1 detects a plurality of anatomical layers in the selected subsetd_(OCT) of the OCT data D_(OCT). The anatomical layers are anatomicallydistinct structures of the eye that overlie each other and may bedistinguished in the depth axis of an OCT image because of differencesin their light diffusive characteristics. The anatomical layers includeanatomical layers that are present in the posterior segment of the eye,including one or more layers of the retina of the eye. Each layer has aninner surface and an outer surface (relative to the vitreous of theeye). The retina can be divided into 10 layers, namely: (1) the innerlimiting membrane (ILM); (2) the nerve fiber layer (NFL); (3) theganglion cell layer (GCL); (4) the inner plexiform layer (IPL); (5) theinner nuclear layer (INL); (6) the outer plexiform layer (OPL); (7) theouter nuclear layer (ONL); (8) the outer limiting membrane (OLM); (9)the photoreceptor layer (PL); and (10) the retinal pigmented epithelium(RPE) monolayer. The supplementary image data generation module 140-1may detect the anatomical layers in the selected subset d_(OCT) of theOCT data D_(OCT) using one of a number of types of algorithm for ocularlayer segmentation known to those skilled in the art, a review of whichis provided in “A Review of Algorithms for Segmentation of OpticalCoherence Tomography from Retina” by R. Kafieh et al, J Med SignalsSens. 2013 January-March; 3(1): 45-60. An anatomical layer segmentationalgorithm is thus used in process S42 to generate layer identificationinformation identifying the boundaries (i.e. an inner surface and anouter surface) of the detected anatomical layers in the selected subsetd_(OCT) of the OCT data D_(OCT), which can be used to identify arespective set of data elements (voxels) in the selected subset d_(OCT)of the OCT data D_(OCT) belonging to each of the detected anatomicallayers.

In process S44 of FIG. 6, the supplementary image data generation module140-1 calculates, for each of the detected anatomical layers, arespective sum value by summing values of data elements (voxels) in thesubset d_(OCT) of the OCT data in the respective anatomical layer. Inother words, the supplementary image data generation module 140-1generates a sum value S_(i) for each anatomical layer L_(i) of the ndetected anatomical layers, L₁, . . . L_(n), where n is an integergreater than or equal to 2, which have been identified using theanatomical layer identification information, by summing values of voxelsthat are located along the A-scans in the subset d_(OCT) of the OCT dataD_(OCT) so as to be between the inner surface and the outer surface ofthe anatomical layer L.

In process S46 of FIG. 6, the supplementary image data generation module140-1 calculates, for each anatomical layer L_(i) of the detectedanatomical layers L₁, . . . L_(n), a respective ratio r_(i) between thesum value S_(i) calculated for the anatomical layer L_(i) and a sum,S_(total), of all the data elements that are in the detected anatomicallayers and in the subset d_(OCT) of the OCT data D_(OCT), i.e. Σ_(i=1)^(n)S_(i).

In process S48 of FIG. 6, the supplementary image data generation module140-1 generates colour information (as the supplementary image dataD_(SI)) based on an ordered sequence of the calculated ratios r₁ tor_(n), wherein the calculated ratios r₁ to r_(n) are arranged in thesame order as the order in which the corresponding anatomical layers arearranged in the posterior segment of the eye. The colour information isgenerated so as to define a colour to be displayed in the combined imagethat uniquely identifies the ordered sequence of the calculated ratiosr₁ to r_(n). The colour displayed in the combined image, at the locationof the designated feature 12, is therefore indicative of the variationof the measured reflectance of the eye in the selected subset d_(OCT) ofthe OCT data D_(OCT) along the depth direction of the retina.

It should be noted that processes S44 to S48 in FIG. 6 need not operateon all of the anatomical layers in the selected subset d_(OCT) of theOCT data D_(OCT) that have been detected in process S42 of FIG. 6, andmay instead operate on at least two of these anatomical layers.Processes S44 to S48 in FIG. 6 operate on one or more detectedanatomical layers forming part of the retina, and may additionallyoperate on one or more anatomical layers that do not form part of theretina, such as a sub-retinal layer which includes the choroid andpathological sub-retinal fluid, for example.

By way of example, in the present example embodiment, a first anatomicallayer L₁, a second anatomical layer L₂, and a third anatomical layer L₃are detected by the supplementary image data generation module 140-1 inprocess S42 of FIG. 6. In process S44 of FIG. 6, the supplementary imagedata generation module 140-1 generates respective sum values S₁, S₂ andS₃ for the anatomical layers L₁, L₂ and L₃, and calculates (in processS46 of FIG. 6) respective ratios r₁, r₂ and r₃ for the anatomical layersL₁, L₂ and L₃. For example, r₁ is calculated as S₁/(S₁+S₂+S₃). Inprocess S48 in FIG. 6 of this example, the supplementary image datageneration module 140-1 generates the colour information by assigning,to each of a red colour component R, a green colour component G and ablue colour component B of the colour to be displayed in the combinedimage, a respective weighting, w_(R), w_(G) or w_(B), for the colourcomponent in accordance with a respective one of the calculated ratiosr₁, r₂ or r₃ in the ordered sequence r₁; r₂; r₃ of the calculatedratios. For instance, in an example where the ratios calculated inprocess S46 of FIG. 6 are r₁=0.1, r₂=0.85 and r₃=0.05, indicating thatanatomical layer L₂ provides a dominant contribution to the measuredreflectance represented by the data element values in the subset d_(OCT)of the OCT data D_(OCT), the assignment of corresponding weightings0.1:0.85:0.05 to the RGB colour components results in the colour that isto be displayed in the combined image being predominantly green, whichindicates to a viewer of the combined image that anatomical layer L₂provides a dominant contribution to the measured reflectance of thedesignated feature 12. In another example, the display of a yellowcolour in the combined image would indicate that anatomical layers L₁and L₂ provide the main contribution to the measured reflectance. In afurther example, the display of a cyan colour in the combined imagewould indicate that layers L₂ and L₃ provide the main contribution tothe measured reflectance.

Referring again to FIG. 5, in process S50, the combined image datageneration module 150 generates the combined image data D_(CI) bycombining the image data D_(F) with the supplementary image data D_(SI),such that a combined image defined by the combined image data D_(CI)provides an indication of the variation at the designated feature 12.The combined image data generation module 150 may be arranged togenerate the combined image data D_(CI) by replacing pixel values of asubset of pixels of the fundus image data D_(F) with pixel values of thesupplementary image data D_(SI), such that a combined image defined bythe combined image data D_(CI) provides an indication of the variationof the measured reflectance of the eye in the selected subset d_(OCT) ofthe OCT data D_(OCT) at the designated feature 12. In exampleembodiments like the present example embodiment, the combined image datageneration module 150 may be arranged to generate the combined imagedata D_(CI) by replacing pixel values of a subset of pixels of thefundus image data D_(F), which subset of pixels defines a subregion ofthe fundus image which is at the location of the designated feature 12in the fundus image 10, with pixel values of the supplementary imagedata D_(SI), such that a combined image defined by the combined imagedata D_(CI) provides an indication of the variation of the measuredreflectance of the eye in the selected subset d_(OCT) of the OCT dataD_(OCT) at the designated feature 12. More specifically, the combinedimage data generation module 150 may, as in the present exampleembodiment, generate the combined image data D_(CI) by modifying theimage data D_(F) such that pixel values in the fundus image 10, whichare at pixel locations corresponding to pixel locations in the OCTen-face image 30 showing data from the selected subset d_(OCT) of theOCT data D_(OCT), are replaced by pixel values representing the colourdetermined by the supplementary image data generation module 140-1 forthat subset d_(OCT) of the OCT data D_(OCT). The aforementionedcorrespondence between pixel locations in the fundus image 10 and theOCT en-face image 30 may be determined by the geometric transformationdiscussed above.

Image A in FIG. 7 is an example of a fundus image 10, which showsseveral features of interest that are highlighted in image B of FIG. 7.The curves bounding the features of interest that are shown in image Bare based on an output from a feature extraction algorithm. Image C ofFIG. 7 is an example of a combined image 40, wherein the regions of thefundus image 10 highlighted in image B are coloured in accordance withthe colour information generated by the supplementary image datageneration module 140-1. In image C, regions 41, 42 and 43 arepredominantly blue, regions 44 and 45 are predominantly red, region 46is predominantly white, and the remaining regions are predominantlygreen.

This colouring indicates that regions 41-43 relate to features of theretina that are nearest the (inner) retinal surface, specifically withinthe inner and outer neurological retina, which is a first anatomicallayer detected in this example. FIG. 8 shows an image of a B-scan takenalong the horizontal line in image C of FIG. 7. Regions of the retina inFIG. 8, which correspond to those in FIG. 7 where the horizontal linecrosses regions 41 and 42, have a higher reflectance near the innersurface of the retina (towards the top of FIG. 8) than the surroundingregions of the retina in FIG. 8.

In FIG. 7, the predominantly green regions relate to features of adeeper retinal layer that is detected in this example, which is theretinal pigment epithelium (RPE) in this example. Regions 44 and 45relate to yet deeper features that are farthest the retinal surface, andlie within the choroid or pathological subretinal fluid, which are inthe third anatomical layer that is detected in this example.

Second Example Embodiment

FIG. 9 is a schematic illustration of an apparatus 300 for processingimage data according to a second example embodiment. The apparatus 300of the second example embodiment differs from the apparatus 100 of thefirst example embodiment by the arrangement of the supplementary imagedata generation module 140-2, which is arranged to generate a differentkind of supplementary image data, D′_(SI). In all other respects, theapparatus 300 of the second example embodiment is the same as apparatus100 of the first example embodiment, as described above.

In the present example embodiment, the supplementary image datageneration module 140-2 processes the selected subset d_(OCT) of the OCTdata D_(OCT) to generate the supplementary image data D′_(SI) by amethod which is a variant of the method described above with referenceto FIG. 6, and which will now be described with reference to FIG. 10.

Processes S42 and S44 in FIG. 10 are the same as processes S42 and S44in FIG. 6, and will therefore not be described again here.

In process S47 of FIG. 10, the supplementary image data generationmodule 140-2 selects, based on the sum values calculated in process S44,an anatomical (e.g. retinal) layer of the detected layers L₁ to L_(n)which provides a dominant contribution to the measured reflectance ofthe among the detected layers for which the sum values have beencalculated. The supplementary image data generation module 140-2 may, asin the present example embodiment, select the anatomical layer thatprovides a dominant contribution by selecting the layer for which thecalculated sum value is largest among the detected layers for which thesum values have been calculated.

In process S49 of FIG. 10, the supplementary image data generationmodule 140-2 generates, as the supplementary image data D′_(SI), graphicimage data defining a graphic which identifies the selected anatomicallayer, for example by a text label naming the layer (in full or in anabbreviated form, for example). The graphic need not such aself-contained identification of the selected anatomical layer, however,and may comprise a pattern, for example, which the viewer can use toidentify the selected anatomical layer by reference to a legend (showingan association between different forms (e.g. patterns) of the graphicand corresponding named anatomical layers) that is also displayed on thescreen.

The combined image data generation module 150 is arranged to generatethe combined image data D′_(CI) by combining the image data D_(F) withthe graphic image data such that the combined image is an annotatedversion of the fundus image 10, wherein the graphic is overlaid on thefundus image 10 (and preferably shaped, for example to have a pointedfeature such as an arrow or the like) so as to indicate the location ofdesignated feature 12. Examples of such an annotated version of thefundus image 10 are shown in FIGS. 11 and 12.

Thus, in some example embodiments like the present example embodiment,the combined image data generation module 150 may be arranged togenerate the combined image data D′_(CI) by replacing pixel values of asubset of pixels of the fundus image data with pixel values of thesupplementary image data D′_(SI), such that a combined image defined bythe combined image data D′_(CI) provides an indication of the variationof the measured reflectance of the eye in the selected subset of the OCTdata at the designated feature 12. The supplementary image datageneration module 140-2 may be arranged to generate supplementary imagedata which defines a graphic that is indicative of the variation, alongthe depth direction of the retina, of the measured reflectance of theeye in the selected subset of the OCT data or, more particularly, howthe measured reflectance of the eye, as indicated in the selected subsetof the OCT data varies along a depth direction of the retina. In exampleembodiments like the present example embodiment, the combined image datageneration module 150 may be arranged to generate the combined imagedata D′_(CI) by replacing pixel values of a subset of pixels of thefundus image data with pixel values of the supplementary image dataD′_(SI) such that a combined image defined by the combined image dataD′_(CI) comprises the graphic, which is overlaid on the fundus image 10so as to provide an indication of the variation of the measuredreflectance of the eye in the selected subset of the OCT data at thedesignated feature 12.

In FIG. 11, the label “INL” is overlaid on the fundus image so as toindicate the location in the fundus image of a feature that is locatedin the inner nuclear layer (INL) of the retina, while the label “GCL” isoverlaid on the fundus image so as to indicate the location in thefundus image of a feature that is located in the ganglion cell layer(GCL) of the retina. The label “None” is overlaid on the fundus imageindicates that the associated feature in the fundus image is not locatedwithin any layer of the retina.

In FIG. 12, the label “RPE” is overlaid on the fundus image so as toindicate the location in the fundus image of a feature that is locatedin the retinal pigment epithelium (RPE) of the retina, while the label“ILM” is overlaid on the fundus image so as to indicate the location inthe fundus image of a feature that is located in the inner limitingmembrane (ILM) that lies between the retina and the vitreous body.

Third Example Embodiment

FIG. 13 is a schematic illustration of an apparatus 400 for processingimage data according to a third example embodiment. The apparatus 400 ofthe third example embodiment differs from the apparatus of the first andsecond example embodiments by the arrangement of the supplementary imagedata generation module 140-3, which is arranged to generate thesupplementary information described in these example embodiments in adifferent way. In all other respects, the apparatus 400 of the thirdexample embodiment is the same as apparatus 100 of the first exampleembodiment and apparatus 300 of the second example embodiment, asdescribed above.

In the present example embodiment, the selected subset d_(OCT) of theOCT data D_(OCT) represents a volumetric image of a part of the retina(possibly in addition to another part of the for example beneath theouter surface of the retina) having a feature of a predetermined typeand is processed by the supplementary image data generation module 140-3to generate the supplementary image data D_(SI) by a method which willnow be described with reference to FIG. 14.

In process S41 of FIG. 14, a model for determining a depth of thefeature of the predetermined type in the depth direction of the retinais trained by supervised learning of examples of OCT data ofpathological regions of at least one other retina. Each of theseexamples of OCT data comprises a single OCT A-scan or two or moreadjacent OCT A-scans, and each of the pathological regions has arespective feature of the predetermined type. While the model is beingtrained, an indication of a respective depth of the respective featurein the depth direction of the retina in each of the examples of OCT datais specified by a user. The model may also be provided with thelocations of retinal layers in the A-scan(s) and, in the trainingprocess, the model may learn to output the depth of the feature whengiven the A-scan (or a group of adjacent A-scans) and the layers asinput.

The depth of the feature may be specified in several ways. The depth isusually defined relative to one or more retinal layers. For example, thedepth may be defined in terms of a number of pixels or as a linearmeasurement (assuming pixel dimensions have been previously estimated)relative to one of the retinal layers. The innermost or outermostretinal surfaces may provide suitable reference points for thismeasurement. Alternatively, the depth may be defined relative tomultiple retinal layers which have been identified by automated means.In this case, the depth may be indicated by the name of the layer thatthe feature is in and, optionally, the displacement of the featurerelative to the inner or outer surface of the layer. Also, the depth maybe indicated by the name of the layer that the feature is in and aunitless normalised displacement of the feature relative to both innerand outer surfaces of the layer, e.g. such that 0 indicates being at theinner surface of the layer and 1 indicates being at the outer surface ofthe layer.

In process S43 of FIG. 14, the supplementary image data generationmodule 140-3 processes the selected subset d_(OCT) of the OCT dataD_(OCT) by using the model that has been trained in process S41 todetermine the depth of the feature in the depth direction of the retina.

In process S45 of FIG. 14, the supplementary image data generationmodule 140-3 generates, as the supplementary image data, either (i)graphic image data defining a graphic which indicates the determineddepth of the feature and is to be overlaid on the fundus image 10 so asto indicate a location of the feature in the combined image 40, or (ii)colour information defining a colour which is to be displayed at alocation of the feature in the combined image and indicates thedetermined depth of the feature. The graphic image data and the colourinformation may be generated as described above.

There has been described, in accordance with example embodiments herein,an apparatus as set out in E1 to E12 below.

-   E1. An apparatus 100 for processing image data defining a fundus    image 10 of a portion of a retina of an eye to include supplementary    information on a designated feature 12 in the fundus image 10, the    apparatus comprising:    -   a feature designation module 110 arranged to designate a feature        in the fundus image 10;    -   a receiver module 120 arranged to receive optical coherence        tomography, OCT, data D_(OCT) of a C-scan 20 of the portion of        the retina;    -   a selection module 130 arranged to select a subset d_(OCT) of        the OCT data D_(OCT) which represents a volumetric image of a        part of the retina at a location on the retina corresponding to        a location of the designated feature 12 in the fundus image 10;    -   a supplementary image data generation module 140-1 arranged to        process the selected subset d_(OCT) of the OCT data D_(OCT) to        generate, as the supplementary information, supplementary image        data D_(SI) indicative of a variation, along a depth direction        of the retina, of a measured reflectance of the eye in the        selected subset d_(OCT) of the OCT data D_(OCT); and    -   a combined image data generation module 150 arranged to generate        combined image data D_(CI) by combining the image data D_(F)        with the supplementary image data D_(SI), such that a combined        image defined by the combined image data D_(CI) provides an        indication of the variation at the designated feature 12.-   E2. The apparatus according to E1, wherein the feature designation    module 110 is arranged to cause the fundus image 10 and a cursor 16    to be displayed on a display 14, such that the cursor 16 can be    controlled to move over the displayed fundus image 10 by a signal    from a user input device 18, and to designate the feature 12 in the    fundus image 10 by recording, in response to a feature designation    command, a value of a first location indicator that is indicative of    a display location of the cursor 16 on the displayed fundus image    10.-   E3. The apparatus according to E2, wherein the selection module 130    is arranged to process the OCT data D_(OCT) to generate an OCT    en-face image 30 of the portion of the retina, cause the OCT en-face    image 30 to be displayed on the display 14 together with the fundus    image 10, such that the cursor 16 can be controlled to move over the    displayed OCT en-face image 30 by the signal from the user input    device 18, and select the subset d_(OCT) of the OCT data D_(OCT)    based on a value of a second location indicator, which is indicative    of a display location of the cursor 16 when the cursor 16 has been    guided by the signal from the user input device 18 to overlay a part    of the displayed OCT en-face image 30 which corresponds to the    designated feature 12 in the displayed fundus image 10.-   E4. The apparatus according to E1, wherein the feature designation    module 110 is arranged to designate the feature 12 in the fundus    image 10 automatically using a feature extraction algorithm.-   E5. The apparatus according to E4, wherein the selection module 130    is arranged to:    -   cause the fundus image 10 and a feature location indicator 19,        which indicates a location of the designated feature 12 in the        fundus image 10, to be displayed on a display 14;    -   process the OCT data D_(OCT) to generate an OCT en-face image 30        of the portion of the retina;    -   cause the OCT en-face image 30 and a cursor 16 to be displayed        on the display 14 together with the fundus image 10, such that        the cursor 16 can be controlled to move over the displayed OCT        en-face image 30 by a signal from a user input device 18; and    -   select the subset d_(OCT) of the OCT data D_(OCT) based on a        value of a second location indicator, which is indicative of a        display location of the cursor 16 when the cursor 16 has been        guided by the signal from the user input device 18 to overlay a        part of the displayed OCT en-face image 30 whose location        corresponds to the location of the designated feature 12 in the        fundus image 10 that is indicated by the feature location        indicator 19.-   E6. The apparatus according to E4, wherein the selection module 130    is arranged to select the subset d_(OCT) of the OCT data D_(OCT) by    applying a geometric transformation, which maps locations in the    fundus image 10 to corresponding A-scan locations in the OCT data    D_(OCT), to the location of the feature 12 in the fundus image 10    which has been designated by the feature extraction algorithm.-   E7. The apparatus according to any of E1 to E6, wherein the feature    12 is one of a dot and a hyper-reflective dot in the fundus image    10, the feature 12 having a pathological cause or being caused by a    reflection from an inner limiting membrane of the retina.-   E8. The apparatus according to E7, wherein the feature 12 has a    pathological cause comprising one of blood leakage, exudation,    drusen, atrophy and/or naevi in the retina, and atrophy of a retinal    pigment epithelium in the retina.-   E9. The apparatus according to any of E1 to E8, wherein the    supplementary image data generation module 140-1 is arranged to    process the selected subset d_(OCT) of the OCT data D_(OCT) to    generate the supplementary image data D_(SI) by:    -   detecting S42 a plurality of anatomical layers of the eye in the        selected subset d_(OCT) of the OCT data D_(OCT);    -   calculating S44, for each of at least two of the detected        anatomical layers, a respective sum value by summing values of        data elements of the subset d_(OCT) of the OCT data D_(OCT) in        the anatomical layer;    -   calculating S46, for each of the at least two of the detected        anatomical layers, a respective ratio between the sum value        calculated for the anatomical layer and a sum of all the data        elements that are in the at least two of the detected anatomical        layers and in the subset d_(OCT) of the OCT data D_(OCT); and    -   generating S48, as the supplementary image data D_(SI), and        based on an ordered sequence of the calculated ratios, wherein        the calculated ratios are arranged in order of the corresponding        anatomical layers in the eye, colour information defining a        colour which is to be displayed in the combined image and        identifies the ordered sequence of the calculated ratios, such        that the colour is indicative of the variation, along the depth        direction of the retina, of the measured reflectance of the eye        in the selected subset d_(OCT) of the OCT data D_(OCT).-   E10. The apparatus according to E9, wherein the supplementary image    data generation module 140-1 is arranged to:    -   detect three anatomical layers in the selected subset d_(OCT) of        the OCT data D_(OCT); and    -   generate the colour information by assigning, to each of a red        colour component, a green colour component and a blue colour        component of the colour to be displayed in the combined image, a        respective weighting for the colour component in accordance with        a respective one of the calculated ratios in the ordered        sequence of the calculated ratios.-   E11. The apparatus according to any of E1 to E8, wherein the    supplementary image data generation module 140-2 is arranged to    process the selected subset d_(OCT) of the OCT data D_(OCT) to    generate the supplementary image data D_(SI) by:    -   detecting a plurality of anatomical layers of the eye in the        selected subset d_(OCT) of the OCT data D_(OCT);    -   calculating, for each anatomical layer of the detected        anatomical layers, a respective sum value by summing values of        data elements of the subset d_(OCT) of the OCT data D_(OCT) in        the anatomical layer;    -   selecting, based on the calculated sum values, an anatomical        layer of the detected anatomical layers which provides a        dominant contribution to the measured reflectance of the eye;        and    -   generating, as the supplementary image data D_(SI), graphic        image data defining a graphic which identifies the selected        anatomical layer, and    -   wherein the combined image data D_(CI) is generated by combining        the image data D_(F) with the graphic image data such that, in        the combined image, the graphic is overlaid on the fundus image        10 at the designated feature 12.-   E12. The apparatus according to any of E1 to E8, wherein the    selected subset d_(OCT) of the OCT data D_(OCT) represents a    volumetric image of a part of the retina having a feature of a    predetermined type, and the supplementary image data generation    module 140-3 is arranged to generate the supplementary image data    D_(SI) by:    -   training a model for determining a depth of the feature of the        predetermined type in the retina, by supervised learning of        examples of OCT data of pathological regions of at least one        other retina, each of the examples of OCT data comprising a        single OCT A-scan or two or more adjacent OCT A-scans, and each        of the pathological regions having a respective feature of the        predetermined type, wherein an indication of a respective depth        of the respective feature in the retina in each of the examples        of OCT data is specified by a user during the training;    -   processing the selected subset door of the OCT data D_(OCT)        using the trained model to determine the depth of the feature in        the retina; and    -   generating, as the supplementary image data, one of        -   graphic image data defining a graphic which indicates the            determined depth of the feature in the retina and is to be            overlaid on the fundus image 10 so as to indicate a location            of the feature in the combined image 40, and        -   colour information defining a colour which is to be            displayed at a location of the feature in the combined image            and indicates the determined depth of the feature in the            retina.

The example aspects described here avoid limitations, specificallyrooted in computer technology, relating to the processing of retinalfundus images. In particular, features of some common eye diseases havesimilar appearance in fundus images, which may be difficult todisambiguate from specular imaging artefacts and the like. By virtue ofthe example aspects described herein, additional information in OCT datamay be leveraged for a clearer rendering of features in fundus images,which can be assimilated by a user without needing to review OCT data,and which can help the user to avoid misinterpretation of artefactualspots and the like in fundus images. Also, by virtue of the foregoingcapabilities of the example aspects described herein, which are rootedin computer technology, the example aspects described herein improvecomputers and computer processing/functionality, and also improve thefield(s) of at least retinal image analysis.

In the foregoing description, example aspects are described withreference to several example embodiments. Accordingly, the specificationshould be regarded as illustrative, rather than restrictive. Similarly,the figures illustrated in the drawings, which highlight thefunctionality and advantages of the example embodiments, are presentedfor example purposes only. The architecture of the example embodimentsis sufficiently flexible and configurable, such that it may be utilized(and navigated) in ways other than those shown in the accompanyingfigures.

Software embodiments of the examples presented herein may be provided asa computer program, or software, such as one or more programs havinginstructions or sequences of instructions, included or stored in anarticle of manufacture such as a machine-accessible or machine-readablemedium, an instruction store, or computer-readable storage device, eachof which can be non-transitory, in one example embodiment. The programor instructions on the non-transitory machine-accessible medium,machine-readable medium, instruction store, or computer-readable storagedevice, may be used to program a computer system or other electronicdevice. The machine- or computer-readable medium, instruction store, andstorage device may include, but are not limited to, floppy diskettes,optical disks, and magneto-optical disks or other types ofmedia/machine-readable medium/instruction store/storage device suitablefor storing or transmitting electronic instructions. The techniquesdescribed herein are not limited to any particular softwareconfiguration. They may find applicability in any computing orprocessing environment. The terms “computer-readable”,“machine-accessible medium”, “machine-readable medium”, “instructionstore”, and “computer-readable storage device” used herein shall includeany medium that is capable of storing, encoding, or transmittinginstructions or a sequence of instructions for execution by the machine,computer, or computer processor and that causes themachine/computer/computer processor to perform any one of the methodsdescribed herein. Furthermore, it is common in the art to speak ofsoftware, in one form or another (e.g. program, procedure, process,application, module, unit, logic, and so on), as taking an action orcausing a result. Such expressions are merely a shorthand way of statingthat the execution of the software by a processing system causes theprocessor to perform an action to produce a result.

Some embodiments may also be implemented by the preparation ofapplication-specific integrated circuits, field-programmable gatearrays, or by interconnecting an appropriate network of conventionalcomponent circuits.

Some embodiments include a computer program product. The computerprogram product may be a storage medium or media, instruction store(s),or storage device(s), having instructions stored thereon or thereinwhich can be used to control, or cause, a computer or computer processorto perform any of the procedures of the example embodiments describedherein. The storage medium/instruction store/storage device may include,by example and without limitation, an optical disc, a ROM, a RAM, anEPROM, an EEPROM, a DRAM, a VRAM, a flash memory, a flash card, amagnetic card, an optical card, nanosystems, a molecular memoryintegrated circuit, a RAID, remote data storage/archive/warehousing,and/or any other type of device suitable for storing instructions and/ordata.

Stored on any one of the computer-readable medium or media, instructionstore(s), or storage device(s), some implementations include softwarefor controlling both the hardware of the system and for enabling thesystem or microprocessor to interact with a human user or othermechanism utilizing the results of the example embodiments describedherein. Such software may include without limitation device drivers,operating systems, and user applications. Ultimately, suchcomputer-readable media or storage device(s) further include softwarefor performing example aspects herein, as described above.

Included in the programming and/or software of the system are softwaremodules for implementing the procedures described herein. In someexample embodiments herein, a module includes software, although inother example embodiments herein, a module includes hardware, or acombination of hardware and software.

While various example embodiments have been described above, it shouldbe understood that they have been presented by way of example, and notlimitation. It will be apparent to persons skilled in the relevantart(s) that various changes in form and detail can be made therein.Thus, the present disclosure should not be limited by any of the abovedescribed example embodiments, but should be defined only in accordancewith the following claims and their equivalents.

Further, the purpose of the Abstract is to enable the Patent Office andthe public generally, and especially the scientists, engineers andpractitioners in the art who are not familiar with patent or legal termsor phraseology, to determine quickly from a cursory inspection thenature and essence of the technical disclosure of the application. TheAbstract is not intended to be limiting as to the scope of the exampleembodiments presented herein in any way. It is also to be understoodthat the procedures recited in the claims need not be performed in theorder presented.

While this specification contains many specific embodiment details,these should not be construed as limiting, but rather as descriptions offeatures specific to particular embodiments described herein. Certainfeatures that are described in this specification in the context ofseparate embodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable sub-combination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

In certain circumstances, multitasking and parallel processing may beadvantageous. Moreover, the separation of various components in theembodiments described above should not be understood as requiring suchseparation in all embodiments, and it should be understood that thedescribed program components and systems can generally be integratedtogether in a single software product or packaged into multiple softwareproducts.

Having now described some illustrative embodiments and embodiments, itis apparent that the foregoing is illustrative and not limiting, havingbeen presented by way of example. In particular, although many of theexamples presented herein involve specific combinations of apparatus orsoftware elements, those elements may be combined in other ways toaccomplish the same objectives. Acts, elements and features discussedonly in connection with one embodiment are not intended to be excludedfrom a similar role in other embodiments or embodiments.

The apparatus and computer programs described herein may be embodied inother specific forms without departing from the characteristics thereof.The foregoing embodiments are illustrative rather than limiting of thedescribed systems and methods. Scope of the apparatus and computerprograms described herein is thus indicated by the appended claims,rather than the foregoing description, and changes that come within themeaning and range of equivalency of the claims are embraced therein.

1. A computer-implemented method of processing image data defining afundus image of a portion of a retina of an eye to include supplementaryinformation on a designated feature in the fundus image, the methodcomprising: designating a feature in the fundus image; receiving opticalcoherence tomography, OCT, data of a C-scan of the portion of theretina; selecting a subset of the OCT data which represents a volumetricimage of a part of the retina at a location on the retina correspondingto a location of the designated feature in the fundus image; processingthe selected subset of the OCT data to generate, as the supplementaryinformation, supplementary image data indicative of a variation, along adepth direction of the retina, of a measured reflectance of the eye inthe selected subset of the OCT data; and generating combined image databy combining the image data with the supplementary image data, such thata combined image defined by the combined image data provides anindication of the variation at the designated feature.
 2. Thecomputer-implemented method according to claim 1, further comprising:causing the fundus image and a cursor to be displayed on a display, suchthat the cursor can be controlled to move over the displayed fundusimage by a signal from a user input device, wherein the feature in thefundus image is designated by recording, in response to a featuredesignation command, a value of a first location indicator that isindicative of a display location of the cursor on the displayed fundusimage.
 3. The computer-implemented method according to claim 2, furthercomprising: processing the OCT data to generate an OCT en-face image ofthe portion of the retina; and causing the OCT en-face image to bedisplayed on the display together with the fundus image, such that thecursor can be controlled to move over the displayed OCT en-face image bythe signal from the user input device, wherein the subset of the OCTdata is selected based on a value of a second location indicator, whichis indicative of a display location of the cursor when the cursor hasbeen guided by the signal from the user input device to overlay a partof the displayed OCT en-face image which corresponds to the designatedfeature in the displayed fundus image.
 4. The computer-implementedmethod according to claim 1, wherein the feature in the fundus image isdesignated automatically by a feature extraction algorithm.
 5. Thecomputer-implemented method according to claim 4, further comprising:causing the fundus image and a feature location indicator, whichindicates the location of the designated feature in the fundus image, tobe displayed on a display; processing the OCT data to generate an OCTen-face image of the portion of the retina; and causing the OCT en-faceimage and a cursor to be displayed on the display together with thefundus image, such that the cursor can be controlled to move over thedisplayed OCT en-face image by a signal from a user input device,wherein the subset of the OCT data is selected based on a value of asecond location indicator, which is indicative of a display location ofthe cursor when the cursor has been guided by the signal from the userinput device to overlay a part of the displayed OCT en-face image whoselocation corresponds to the location of the designated feature in thefundus image that is indicated by the feature location indicator.
 6. Thecomputer-implemented method according to claim 4, wherein the subset ofthe OCT data is selected by applying a geometric transformation, whichmaps locations in the fundus image to corresponding A-scan locations inthe OCT data, to the location of the feature in the fundus image whichhas been designated by the feature extraction algorithm.
 7. Thecomputer-implemented method according to claim 1, wherein the feature isone of a dot and a hyper-reflective dot in the fundus image, the featurehaving a pathological cause or being caused by a reflection from aninner limiting membrane of the retina.
 8. The computer-implementedmethod according to claim 7, wherein the feature has a pathologicalcause comprising one of blood leakage, exudation, drusen, atrophy and/ornaevi in the retina, and atrophy of a retinal pigment epithelium in theretina.
 9. The computer-implemented method according to claim 1, whereinthe selected subset of the OCT data is processed to generate thesupplementary image data by: detecting a plurality of anatomical layersof the eye in the selected subset of the OCT data, the anatomical layerscomprising one or more retinal layers of the retina; calculating, foreach of at least two of the detected anatomical layers, a respective sumvalue by summing values of data elements of the subset of the OCT datain the anatomical layer; calculating, for each of the at least two ofthe detected anatomical layers, a respective ratio between the sum valuecalculated for the anatomical layer and a sum of all the data elementsthat are in the at least two of the detected anatomical layers and inthe subset of the OCT data; and generating, as the supplementary imagedata, and based on an ordered sequence of the calculated ratios, whereinthe calculated ratios are arranged in order of the correspondinganatomical layers of the eye, colour information defining a colour whichis to be displayed in the combined image and identifies the orderedsequence of the calculated ratios, such that the colour is indicative ofthe variation, along the depth direction of the retina, of the measuredreflectance of the eye in the selected subset of the OCT data.
 10. Thecomputer-implemented method according to claim 9, wherein threeanatomical layers in the selected subset of the OCT data are detected,and the colour information is generated by assigning, to each of a redcolour component, a green colour component and a blue colour componentof the colour to be displayed in the combined image, a respectiveweighting for the colour component in accordance with a respective oneof the calculated ratios in the ordered sequence of the calculatedratios.
 11. The computer-implemented method according to claim 1,wherein the selected subset of the OCT data is processed to generate thesupplementary image data by: detecting a plurality of anatomical layersof the eye in the selected subset of the OCT data, the anatomical layerscomprising one or more retinal layers of the retina; calculating, foreach anatomical layer of the detected anatomical layers, a respectivesum value by summing values of data elements of the subset of the OCTdata in the anatomical layer; selecting, based on the calculated sumvalues, an anatomical layer of the detected anatomical layers whichprovides a dominant contribution to the measured reflectance of the eye;and generating, as the supplementary image data, graphic image datadefining a graphic which identifies the selected anatomical layer. 12.The computer-implemented method according to claim 1, wherein theselected subset of the OCT data represents a volumetric image of a partof the retina having a feature of a predetermined type and is processedto generate the supplementary image data by: training a model fordetermining a depth of the feature of the predetermined type in thedepth direction of the retina, by supervised learning of examples of OCTdata of pathological regions of at least one other retina, each of theexamples of OCT data comprising a single OCT A-scan or two or moreadjacent OCT A-scans, and each of the pathological regions having arespective feature of the predetermined type, wherein an indication of arespective depth of the respective feature in the depth direction of theretina in each of the examples of OCT data is specified by a user duringthe training; processing the selected subset of the OCT data using thetrained model to determine the depth of the feature in the depthdirection of the retina; and generating, as the supplementary imagedata, one of graphic image data defining a graphic which indicates thedetermined depth of the feature and is to be overlaid on the fundusimage so as to indicate a location of the feature in the combined image,and colour information defining a colour which is to be displayed at alocation of the feature in the combined image and indicates thedetermined depth of the feature.
 13. A computer program comprisingcomputer program instructions which, when executed by a computerprocessor, cause the computer processor to perform a method ofprocessing image data defining a fundus image of a portion of a retinaof an eye to include supplementary information on a designated featurein the fundus image, the method comprising: designating a feature in thefundus image; receiving optical coherence tomography, OCT, data of aC-scan of the portion of the retina; selecting a subset of the OCT datawhich represents a volumetric image of a part of the retina at alocation on the retina corresponding to a location of the designatedfeature in the fundus image; processing the selected subset of the OCTdata to generate, as the supplementary information, supplementary imagedata indicative of a variation, along a depth direction of the retina,of a measured reflectance of the eye in the selected subset of the OCTdata; and generating combined image data by combining the image datawith the supplementary image data, such that a combined image defined bythe combined image data provides an indication of the variation at thedesignated feature.
 14. A non-transitory computer-readable storagemedium storing the computer program according to claim
 13. 15. Anapparatus for processing image data defining a fundus image of a portionof a retina of an eye to include supplementary information on adesignated feature in the fundus image, the apparatus comprising: afeature designation module arranged to designate a feature in the fundusimage; a receiver module arranged to receive optical coherencetomography, OCT, data of a C-scan of the portion of the retina; aselection module arranged to select a subset of the OCT data whichrepresents a volumetric image of a part of the retina at a location onthe retina corresponding to a location of the designated feature in thefundus image; a supplementary image data generation module arranged toprocess the selected subset of the OCT data to generate, as thesupplementary information, supplementary image data indicative of avariation, along a depth direction of the retina, of a measuredreflectance of the eye in the selected subset of the OCT data; and acombined image data generation module arranged to generate combinedimage data by combining the image data with the supplementary imagedata, such that a combined image defined by the combined image dataprovides an indication of the variation at the designated feature.